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8.4.2 General Setting and Definitions 

Example 8.22 provided a basic introduction to hypothesis testing. Here, we would like to provide a general 
setting for problems of hypothesis testing and formally define the terminology that is used in hypothesis 
testing. Although there are several new phrases such as null hypothesis, type I error, significance level, 
etc., there are not many new concepts or tools here. Thus, after going through a few examples, the concepts 
should become clear. 

Suppose that 9 is an unknown parameter. A hypothesis is a statement such as 9 — 1,9 > 1.3, 

9^ 0.5 , etc. In hypothesis testing problems, we need to decide between two contradictory hypotheses. 
More precisely, let S be the set of possible values for 9. Suppose that we can partition S into two disjoint 
sets Sq and S\ . Let Hq be the hypothesis that 9 G Sq , and let H\ be the hypothesis that 9 E <Si. 


Hq (the null hypothesis): 9 £ Sq . 

H\ (the alternative hypothesis): 9 €. S\. 

In Example 8.22 . S — [0,1], Sq — and (Si = [0, 1] — Here, Hq is an example of a 

simple hypothesis because Sq contains only one value of 9. On the other hand, H \ is an example of 
composite hypothesis since S\ contains more than one element. It is often the case that the null hypothesis 
is chosen to be a simple hypothesis. 

Often, to decide between Hq and H i , we look at a function of the observed data. For instance, in 
Example 8.22 . we looked at the random variable Y, defined as 

Y __ X-n9 0 

V n6> o(l — #0 ) 


where X was the total number of heads. Here, X is a function of the observed data (sequence of heads 
and tails), and thus Y is a function of the observed data. We call Y a statistic. 

Definition 8.3. Let X\. X 2 , • • •, X n be a random sample of interest. A statistic is a real-valued function 
of the data. For example, the sample mean, defined as 


W(X 1 ,X 2 ,---,X n ) 


Xi + v 2 +... +x n 

n 


is a statistic. A test statistic is a statistic based on which we build our test. 


To decide whether to choose Hq or H 1 , we choose a test statistic, W=W{X 1 ,X 2 ,---,X n ). 

Now, assuming Hq, we can define the set A C M as the set of possible values of W for which we would 
accept Hq . The set A is called the acceptance region, while the set R = M — A is said to be the 
rejection region. In Example 8.22 . the acceptance region was found to be the set A — [—1.96,1.96], 
and the set R = ( — OO, —1.96) U (1.96, 00 ) was the rejection region. 
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There are two possible errors that we can make. We define type I error as the event that we reject Hq 
when Hq is true. Note that the probability of type I error in general depends on the real value of 9. More 
specifically, 


P(type I error | 9) — P(Reject Hq \ 9) 

= P{W e R\9), for 9 £ Sq. 

If the probability of type I error satisfies 

P(type I error) < a, for all 9 E Sq, 

then we say the test has significance level O or simply the test is a level O test. Note that it is often the 
case that the null hypothesis is a simple hypothesis, so Sq has only one element (as in Example 8.22 1. The 
second possible error that we can make is to accept Hq when Hq is false. This is called the type II error. 
Since the alternative hypothesis, H \, is usually a composite hypothesis (so it includes more than one value 
of 9), the probability of type II error is usually a function of 9. The probability of type II error is usually 
shown by f3\ 


f3(9) — P(Accept Ho \ 9), for 9 E Si. 

We now go through an example to practice the above concepts. 


Example 8.23 

Consider a radar system that uses radio waves to detect aircraft. The system receives a signal and, based on 
the received signal, it needs to decide whether an aircraft is present or not. Let X be the received signal. 
Suppose that we know 


x = w, if no aircraft is present. 

X = 1 + w, if an aircraft is present. 

wherein ~ N{ 0,(J 2 = |-).Thus, we can write X = 9 + W, where 9 = 0 if there is no aircraft, 
and 9 = 1 if there is an aircraft. Suppose that we define Hq and H\ as follows: 


Hq (null hypothesis): No aircraft is present. 

Hi (alternative hypothesis): An aircraft is present. 

a. Write the null hypothesis, Hq, and the alternative hypothesis, H i, in terms of possible values of 9. 

b. Design a level 0.05 test (O = 0.05) to decide between Hq and Hi. 

c. Find the probability of type II error, p, for the above test. Note that this is the probability of missing 
a present aircraft. 

d. If we observe X = 0.6, is there enough evidence to reject Hq at significance level O = 0.01? 

e. If we would like the probability of missing a present aircraft to be less than 5%, what is the smallest 
significance level that we can achieve? 

• Solution 
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a. The null hypothesis corresponds to 9 — 0 and the alternative hypothesis corresponds to 
9 — 1 . Thus, we can write 


H o (null hypothesis): No aircraft is present: 9 — 0. 

H\ (alternative hypothesis): An aircraft is present: 9 — 1. 
Note that here both hypotheses are simple. 


b. To decide between H 0 and Hi , we look at the observed data. Here, the situation is 
relatively simple. The observed data is just the random variable X. Under Hq, 

X ~ iV(0,|) , and under H\ , X ~ _/V(l. ^-). Thus, we can suggest the following 
test: We choose a threshold C. If the observed value of X is less than C, we choose Ho 
(i.e., 9 — EX — 0). If the observed value of X is larger than C, we choose H\ (i.e., 
9 — EX — 1). To choose C, we use the required Ot : 


P(type I error) 


P(Reject Ho \ H 0 ) 
P{X > c | Ho) 
P{W > c ) 


1 — $(3c) (since assuming Ho,X 

9 


Letting P(type I error) = Ct, we obtain 


c 



a). 


Letting Ot = 0.05, we obtain 


C = 


i^-^O^S) = 0.548 


c. Note that, here, the alternative hypothesis is a simple hypothesis. That is, it includes 
only one value of 9 (i.e., 9 — 1). Thus, we can write 

j3 — P(type II error) = P(accept Hq \ Hi) 

= P{X <c \ Hi) 

= P{l + W < c) 

= P{W < c-1) 

= $(3(c-l)). 

Since C = 0.548, we obtain/3 = 0.088. 

d. In part (b), we obtained 
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c = —$ 1 (1 — a). 

3 v ’ 

ForCt = 0.01, we have C = |-4> _1 (0.99) = 0.775 which is larger than 0.6. 
Thus, we cannot reject H g at significance level Oi = 0.01. 

e. In part (c), we obtained 

/? = 5(3(0-1)). 

To have f3 — 0.05, we obtain 

c = 1 + l$- 1 (/3) 

= l + i$- 1 (0.05) 

3 

= 0.452 

Thus, we need to have c < 0.452 to obtain (3 < 0.05- Therefore, 

P(type I error) = 1 — 4>(3c) 

= l- $(3 x 0.452) 

= 0.0875, 

which means that the smallest significance level that we can achieve is Oi = 0.0875. 


Trade-off Between Oi and f3: Since Oi and (3 indicate error probabilities, we would ideally like both of 
them to be small. However, there is in fact a trade-off between Oi and f3. That is, if we want to decrease the 
probability of type I error (Ct), then the probability of type II error (J3) increases, and vise versa. To see 
this, we can look at our analysis in Example 8.23 . In that example, we found 

a = 1 — 4>(3c), 

0 = <&(3(c — 1)). 

Note that 4* (.X‘) is an increasing function. If we make C larger, Ct becomes smaller, and f3 becomes larger. 
On the other hand, if we make C smaller, Oi becomes larger, and (3 becomes smaller. Figure 8.10 shows 
type I and type II error probabilities for Example 8.23 . 


PDF of X under 
H o :N(0,9) 


PDF of X under 
H r :N( 1,9) 



0 <= \ X 
P = p(X < c\Hi) a = p{X > c\H 0 ) 

Figure 8.10 - Type I and type II errors in Example 8.23 . 
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